{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "adopted-tooth",
   "metadata": {},
   "source": [
    "# Italian Fiscal Codes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "annual-musician",
   "metadata": {},
   "source": [
    "## Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "original-signature",
   "metadata": {},
   "source": [
    "The function `clean_it_codicefiscale()` cleans a column containing Italian fiscal codes (Codice Fiscale) strings, and standardizes them in a given format. The function `validate_it_codicefiscale()` validates either a single Codice Fiscale strings, a column of Codice Fiscale strings or a DataFrame of Codice Fiscale strings, returning `True` if the value is valid, and `False` otherwise."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "governing-short",
   "metadata": {},
   "source": [
    "Codice Fiscale strings can be converted to the following formats via the `output_format` parameter:\n",
    "\n",
    "* `compact`: only number strings without any seperators or whitespace, like \"RCCMNL83S18D969H\"\n",
    "* `standard`: Codice Fiscale strings with proper whitespace in the proper places. Note that in the case of Codice Fiscale, the compact format is the same as the standard one.\n",
    "* `birthdate`: split the date parts from the number and return the birth date, like \"1875-03-16\".\n",
    "* `gender`: get the person's birth gender ('M' or 'F').\n",
    "\n",
    "Invalid parsing is handled with the `errors` parameter:\n",
    "\n",
    "* `coerce` (default): invalid parsing will be set to NaN\n",
    "* `ignore`: invalid parsing will return the input\n",
    "* `raise`: invalid parsing will raise an exception\n",
    "\n",
    "The following sections demonstrate the functionality of `clean_it_codicefiscale()` and `validate_it_codicefiscale()`. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "blind-ridge",
   "metadata": {},
   "source": [
    "### An example dataset containing Codice Fiscale strings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "silent-south",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "df = pd.DataFrame(\n",
    "    {\n",
    "        \"codicefiscale\": [\n",
    "            'RCCMNL83S18D969H',\n",
    "            'RCCMNL83S18D969',\n",
    "            '7542011030',\n",
    "            '7552A10004',\n",
    "            '8019010008',\n",
    "            \"hello\",\n",
    "            np.nan,\n",
    "            \"NULL\",\n",
    "        ], \n",
    "        \"address\": [\n",
    "            \"123 Pine Ave.\",\n",
    "            \"main st\",\n",
    "            \"1234 west main heights 57033\",\n",
    "            \"apt 1 789 s maple rd manhattan\",\n",
    "            \"robie house, 789 north main street\",\n",
    "            \"1111 S Figueroa St, Los Angeles, CA 90015\",\n",
    "            \"(staples center) 1111 S Figueroa St, Los Angeles\",\n",
    "            \"hello\",\n",
    "        ]\n",
    "    }\n",
    ")\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "shared-climate",
   "metadata": {},
   "source": [
    "## 1. Default `clean_it_codicefiscale`\n",
    "\n",
    "By default, `clean_it_codicefiscale` will clean codicefiscale strings and output them in the standard format with proper separators."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "twelve-gibraltar",
   "metadata": {},
   "outputs": [],
   "source": [
    "from dataprep.clean import clean_it_codicefiscale\n",
    "clean_it_codicefiscale(df, column = \"codicefiscale\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dedicated-banks",
   "metadata": {},
   "source": [
    "## 2. Output formats"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "brilliant-constant",
   "metadata": {},
   "source": [
    "This section demonstrates the output parameter."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "reserved-actor",
   "metadata": {},
   "source": [
    "### `standard` (default)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "reliable-profit",
   "metadata": {},
   "outputs": [],
   "source": [
    "clean_it_codicefiscale(df, column = \"codicefiscale\", output_format=\"standard\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "prerequisite-declaration",
   "metadata": {},
   "source": [
    "### `compact`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "caroline-ground",
   "metadata": {},
   "outputs": [],
   "source": [
    "clean_it_codicefiscale(df, column = \"codicefiscale\", output_format=\"compact\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "short-blend",
   "metadata": {},
   "source": [
    "### `birthdate`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "elementary-theta",
   "metadata": {},
   "outputs": [],
   "source": [
    "clean_it_codicefiscale(df, column = \"codicefiscale\", output_format=\"birthdate\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "caroline-bruce",
   "metadata": {},
   "source": [
    "### `gender`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "stone-integer",
   "metadata": {},
   "outputs": [],
   "source": [
    "clean_it_codicefiscale(df, column = \"codicefiscale\", output_format=\"gender\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "current-pathology",
   "metadata": {},
   "source": [
    "## 3. `inplace` parameter\n",
    "\n",
    "This deletes the given column from the returned DataFrame. \n",
    "A new column containing cleaned Codice Fiscale strings is added with a title in the format `\"{original title}_clean\"`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "divine-winter",
   "metadata": {},
   "outputs": [],
   "source": [
    "clean_it_codicefiscale(df, column=\"codicefiscale\", inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "forbidden-bumper",
   "metadata": {},
   "source": [
    "## 4. `errors` parameter"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "posted-saying",
   "metadata": {},
   "source": [
    "### `coerce` (default)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bizarre-plaza",
   "metadata": {},
   "outputs": [],
   "source": [
    "clean_it_codicefiscale(df, \"codicefiscale\", errors=\"coerce\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "organized-alignment",
   "metadata": {},
   "source": [
    "### `ignore`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "acute-dallas",
   "metadata": {},
   "outputs": [],
   "source": [
    "clean_it_codicefiscale(df, \"codicefiscale\", errors=\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "welcome-yesterday",
   "metadata": {},
   "source": [
    "## 4. `validate_it_codicefiscale()`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "short-albuquerque",
   "metadata": {},
   "source": [
    "`validate_it_codicefiscale()` returns `True` when the input is a valid Codice Fiscale. Otherwise it returns `False`.\n",
    "\n",
    "The input of `validate_it_codicefiscale()` can be a string, a Pandas DataSeries, a Dask DataSeries, a Pandas DataFrame and a dask DataFrame.\n",
    "\n",
    "When the input is a string, a Pandas DataSeries or a Dask DataSeries, user doesn't need to specify a column name to be validated. \n",
    "\n",
    "When the input is a Pandas DataFrame or a dask DataFrame, user can both specify or not specify a column name to be validated. If user specify the column name, `validate_it_codicefiscale()` only returns the validation result for the specified column. If user doesn't specify the column name, `validate_it_codicefiscale()` returns the validation result for the whole DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "played-christianity",
   "metadata": {},
   "outputs": [],
   "source": [
    "from dataprep.clean import validate_it_codicefiscale\n",
    "print(validate_it_codicefiscale('RCCMNL83S18D969H'))\n",
    "print(validate_it_codicefiscale('RCCMNL83S18D969'))\n",
    "print(validate_it_codicefiscale('7542011030'))\n",
    "print(validate_it_codicefiscale('7552A10004'))\n",
    "print(validate_it_codicefiscale('8019010008'))\n",
    "print(validate_it_codicefiscale(\"hello\"))\n",
    "print(validate_it_codicefiscale(np.nan))\n",
    "print(validate_it_codicefiscale(\"NULL\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "congressional-booth",
   "metadata": {},
   "source": [
    "### Series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "subtle-operations",
   "metadata": {},
   "outputs": [],
   "source": [
    "validate_it_codicefiscale(df[\"codicefiscale\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "minus-controversy",
   "metadata": {},
   "source": [
    "### DataFrame + Specify Column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "union-evaluation",
   "metadata": {},
   "outputs": [],
   "source": [
    "validate_it_codicefiscale(df, column=\"codicefiscale\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "three-exclusive",
   "metadata": {},
   "source": [
    "### Only DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "characteristic-fairy",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "validate_it_codicefiscale(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "chinese-siemens",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
